System and method for preoperative planning for total hip arthroplasty

ABSTRACT

Planning tools for surgery, particularly for THA, are provided. Images of musculoskeletal structure of a patient (e.g. associated with respective planes and in a same or different functional position) may be displayed together and via co-registration and spatial transformations, 3D implants or other objects may be rendered and overlaid in a same position correctly with respect to each image. The 3D implant may be fit (e.g. via handles or automatically using image processing) to an existing implant in the patient and moved to other positions, for example, to measure the existing position or plan for an initial or new position. Measures may be represented with respect to various planes associated with the respective image and/or with respect to an existing implant.

CROSS-REFERENCE

The present application is a continuation of prior U.S. patentapplication Ser. No. 16/754,020 filed Oct. 6, 2020, and entitled “SystemAnd Method For Preoperative Planning For Total Hip Arthroplasty”. U.S.patent application Ser. No. 16/754,020 application claims the domesticbenefit of the following applications: 1) U.S. Provisional ApplicationNo. 62/569,106 filed Oct, 6, 2017 and entitled “System and Method forPreoperative Planning for Total Hip Arthroplasty” and 2) U.S.Provisional Application No. 62/691,912 filed Jun. 29, 2018 and entitled“System and Method for Preoperative Planning for Total HipArthroplasty”. The entire contents of each application are incorporatedherein by reference.

FIELD

The subject matter relates to computer assisted procedures in surgeryincluding methods and systems therefor and more particularly to a systemand method for preoperative planning such as for total hip arthroplasty.

BACKGROUND

This specification and the associated drawings will use hip replacementsurgery as the primary example; this example is intended on beingnon-limiting, and the systems and methods described herein may beapplied to various other surgical procedures or musculoskeletaldiagnoses.

Hip replacement surgery (or Total Hip Arthroplasty, THA) involvesreplacing a patient's native hip (proximal femur and acetabulum) withprosthetic implants. The position of the implants is important for thefunction of the new joint. For example, a misaligned acetabular cup maydislocate post-operatively, and a malpositioned femur stem may causeperiprosthetic fracture or leg length inequality. When referring to animplanted prosthesis, the term “position” may refer to clinicallyrelevant parameters such as translational position relative to a bone orangular position relative to anatomical axes; the term “position” isintended to be interpreted based on the context of its use by oneskilled in the art. In a hip replacement surgery, the position ofindividual implants is important, as is the combined position of bothsides of the prosthetic joint (for example, the combined anteversion ofa hip joint is the sum of the acetabular and femoral anteversion angles,and is known to be clinically relevant for hip joint stability).

The ideal position of joint implants may be patient specific, and inparticular, dependent on a patient's functional musculoskeletalkinematics (i.e. how the various musculoskeletal structures move throughfunctional positions). A functional position or movement generally meansthe biomechanics and kinematics of a musculoskeletal system duringbasic, everyday tasks (particularly those tasks essential to maintaininga basic quality of life, such as standing up, sitting down, mountingstairs, etc.). For example, in a healthy musculoskeletal system, thepelvis tilts in the sagittal plane between a standing and sittingposition. Patients with abnormal musculoskeletal systems (e.g.hypermobile spine, degenerative disease and/or fused lumbar spines) mayexperience abnormal functional musculoskeletal kinematics, such as anabnormal degree of tilting in the sagittal plane from standing tositting. For example, a stiff spine may cause the pelvis to have arelatively low change in tilt (posteriorly) when sitting, which resultsin abnormally high hip flexion, putting the patient at risk ofimpingement and/or dislocation. FIGS. 1A and 1B illustrate normal pelvicmovement from the standing to the sitting position and FIGS. 2A and 2Billustrate abnormal pelvic movement from the standing to the sittingposition, where the abnormality is the result of spinal stiffness, asillustrated by the spinal hardware. In addition to affecting the changein tilt between functional positions, deformity may also cause themusculoskeletal system position to deviate form a nominal healthyposition (e.g. an unhealthy patient may have a pelvis tilted in thesagittal and/or coronal planes when standing normally).

SUMMARY

Disclosed are methods and systems to provided planning tools forsurgery, particularly for THA. Images of musculoskeletal structure of apatient (e.g. associated with respective planes and in a same ordifferent functional position) may be displayed together and viaco-registration and spatial transformations, 3D implants or otherobjects may be rendered and overlaid in a same position correctly withrespect to each image. The 3D implant may be fit (e.g. via handles orautomatically using image processing) to an existing implant in thepatient and moved to other positions, for example, to measure theexisting position or plan for an initial or new position. Measures maybe represented with respect to various planes associated with therespective image and/or with respect to an existing implant.

There is provided a computer implemented method comprising: accessingand displaying a standing AP image, a standing lateral image, and asitting lateral image of a musculoskeletal structure of a patient;receiving respective axes coordinates on the standing AP image andstanding lateral image respectively as input, and defining referenceaxes of the musculoskeletal structure on the standing AP image and thestanding lateral image based on the respective axes coordinates;receiving respective tilt coordinates on the standing lateral image andthe sitting lateral image respectively as input, and determining achange in tilt parameter between the standing AP image and the standinglateral image of the musculoskeletal structure based on the respectivetilt coordinates; determining a spatial transformation between thestanding AP image and the standing lateral image and a spatialtransformation between the standing lateral image and the sittinglateral image; rendering and overlaying a 3D implant for each of thestanding AP image and the standing lateral image in a first positionrelative to the reference axes based on the reference axes and thespatial transformation between the standing AP image and the standinglateral image; rendering and overlaying the 3D implant in the firstposition for the sitting lateral image based on the change in tiltparameter and the spatial transformation between the standing lateralimage and the sitting lateral image; receiving input representing asecond position of the 3D implant; and rendering and overlaying the 3Dimplant in the second position using each space transformation and thechange in tilt parameter to update, in real time, each of the standingAP image, the standing lateral image and the sitting lateral image.

In the computer implemented method of claim 1 the images may be of anyof the following modalities: x-ray, CT, MRI, EOS.

The computer implemented method may further comprise displaying changein tilt parameter.

In the computer implemented method, determining spatial transformationsmay comprise one or more of: performing computations based at least inpart on the received coordinates; performing computations based oncorresponding common features between received images; and performingcomputations based on known image acquisition spatial information.

In the computer implemented method the musculoskeletal structure maycomprise a pelvis and the method may comprise planning for a total hiparthroplasty (THA) procedure.

In the computer implemented method the musculoskeletal structure may bea pelvis, the 3D implant may be an acetabular cup and the first positionmay be 40 degrees of inclination and 15 degrees of anteversion.

In the computer implemented method the first position may be based, atleast in part, on the change in tilt parameter, and may be selected tomaximize stability through functional patient movements.

The computer implemented method may further comprise: receivingtemplating information comprising one or more of implant size anddesired location; and rendering the 3D implant according to at least asubset of the received templating information.

The computer implemented method may further comprise displaying aposition information representing a current position of the 3D implant.In the computer implemented method the 3D implant may be an acetabularcup and the position information may be an inclination angle and ananteversion angle.

In the computer implemented method, the input representing the secondposition may be based on one or more of: a manipulation of an imageassociated with the 3D implant as overlaid on a one of the standing APimages, standing lateral image and sitting lateral image; and aninputting of numerical values. In the computer implemented method themanipulation of the image may comprise clicking and dragging a handleassociated with the 3D implant overlay.

In the computer implemented method the images associated with the 3Dimplant and associated renderings may be provided for simultaneousdisplay.

There is provided a computer implemented method comprising: accessingand displaying at least three images of a musculoskeletal structure of apatient, where one pair of images are of different views of a firstposition, and another pair of images are of the same view in differentfunctional positions; defining reference axes of the musculoskeletalstructure on at least two images, based on user input; determining apositional change parameter of the musculoskeletal structure based onthe two images of a common view in different functional positions, thepositional change parameter representing the positional change of thepatient between the functional positions; determining spatialtransformations between the images; rendering and overlaying a 3Dimplant for each image in a first position relative to the referenceaxes, and according to the respective spatial transformations, andoverlaying the 3D implant renderings on the respective images; receivinginput representing a second position; and updating, in at least nearreal time, the rendering and overlay accordingly for each image.

Complimentary computing device and computer program product aspects aswell as other aspects and features will be apparent to one of ordinaryskill in the art.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate normal pelvic movement from the standing tothe sitting position in accordance with the prior art.

FIGS. 2A and 2B illustrate abnormal pelvic movement from the standing tothe sitting position in accordance with the prior art.

FIGS. 3-6 each illustrates a portion of a graphical user interface (GUI)configured to display images of an anatomy (e.g. the same anatomy fromdifferent views or in different positions) and enabled to receive inputto define coordinates from positions on the screen that are associatedto positions on the anatomy shown in the images.

FIGS. 7 and 8 illustrate a GUI showing multiple images (e.g. x-rays) ofa same anatomy simultaneously where the GUI is configured to receiveinput to define coordinates from positions on the screen that areassociated to positions on the anatomy, among other features.

FIG. 9 is a block diagram of a computing device configured in accordancewith the teachings herein.

FIGS. 10 and 11 are flowchart of respective operations of a computingdevice in accordance with the teachings herein.

FIGS. 12A and 12B are images of an anatomy from two views showingreference axes in each view.

FIGS. 13 and 14 are illustrations of a portion of a respective GUI 1300showing two images and enabled to receive input of coordinates anddetermine (e.g. compute) clinically relevant values therefrom, amongother features.

FIG. 15 is a flowchart of operations of a computing device in accordancewith the teachings herein.

FIGS. 16A-16C are illustrations of portions or respective GUIs showingrespective images and enabled to receive input of coordinates anddetermine (e.g. compute) clinically relevant values therefrom, amongother features.

FIG. 17 is an illustration of a portion of GUI showing one of the imagesthat may be displayed by the GUI enabled to receive input of coordinatesand determine (e.g. compute) clinically relevant values therefrom, amongother features.

FIG. 18 is an illustration of a portion of a GUI showing four respectiveimages enabled to receive input of coordinates and determine (e.g.compute) clinically relevant values therefrom, among other features andFIG. 19 is an enlarged view of a portion of the GUI of FIG. 18 in whicha control is invoked.

FIGS. 20-22 are flowcharts of operations of a computing device inaccordance with the teachings herein.

FIG. 23 is a block diagram representing skew error and a transformationto correct skew error.

FIGS. 24-26 are illustrations of a portion of a GUI showing fourrespective images enabled to receive input of coordinates and determine(e.g. compute) clinically relevant values therefrom, and to display asafe zone indicator.

FIGS. 27A and 27B are illustrations showing a safe zone defined as a 3Dsector of a sphere (e.g. defined relative to the angles and ranges ofinclination and anteversion.

FIG. 28 is a flowchart of operations of a computing device in accordancewith the teachings herein.

FIGS. 29-31 are illustrations of a portion of a GUI 2900 showing animage 2902 of post-operative anatomy.

FIG. 32 is an enlarged view of a portion of the GUI of FIG. 31 .

FIGS. 33A and 33B are illustrations of cup implants as hemispheres toshow calculations of inclination and anteversion delta measurements.

FIG. 34 is a flowchart of operations of a computing device in accordancewith the teachings herein.

DESCRIPTION

A computer-implemented method may help surgeons or other users to plansurgery with the goal of determining a desired or ideal implantposition, based on functional musculoskeletal kinematics. Thecomputer-implemented method may access a plurality of functional medicalimages, define reference axes in the images, determine the spatialtransformations between the plurality of images, render a 3D model of animplant for each image according to the spatial transformations, anddisplay the images along with an overlay based on the rendered 3D modelcorresponding to each image.

The functional medical images may include at least two images ofdifferent views of the patient's musculoskeletal region of interest. Forexample, an anterior-posterior (AP) view (i.e. the image planecorresponds to the patient's coronal plane), and a lateral view (i.e.the image plane corresponds with the patient's sagittal plane). Thefunctional medical images may include at least two images in which thepatient's musculoskeletal region of interest is in two differentfunctional positions (e.g. standing and sitting).

The functional medical images may be of any modality wheremusculoskeletal structures can be visualized. For example, x-rays,computed tomography (CT) scans, magnetic resonance imaging (MRI) scans,EOS scans (EOS Imaging, Paris, France), etc. The medical images may bein DICOM, or any other digital format. All images may be of a singlemodality; alternatively, some images may be of differing modalities.

The functional medical images are related by spatial transformations. Aspatial transformation between two images is the relative change in poseof the image reference frames with respect to the musculoskeletal regionof interest. For example, the image reference frame of a pure AP view ofa structure, and the image reference frame of a pure lateral view of thesame structure mean the spatial transformation between the respectiveimages is a rotation of 90° about the patient's longitudinal axis. Themethod of image acquisition, including the imaging modality, imagingequipment and imaging protocol, influences the spatial transformations.For example, some imaging equipment offers simultaneous perpendicularimaging capabilities. In this case the spatial transformation may bedetermined based on the imaging equipment. Parallax or other types ofdistortion may be inherent in certain imaging modalities; the respectivespatial transformations may take into account the effects of suchdistortions.

In THA, the functional musculoskeletal kinematics of a patient mayinfluence the ideal alignment of the acetabular cup implant. Inparticular, sagittal pelvic movement (or lack thereof) betweenfunctional positions may influence how the acetabular cup should beoriented to minimize risk of impingement and/or dislocation. The changein pelvic position in the sagittal plane may be measured by determiningthe pelvic orientation difference between two lateral x-ray images indifferent functional positions (e.g. standing and sitting). The changein pelvic position in the sagittal plane is also referred to as thechange in pelvic tilt. Standing lateral x-rays of the musculoskeletalregion of interest (i.e. the pelvis and proximal femur) may be performedin THA as part of routine diagnosis and planning; other functionalimages of the pelvis may also be obtained through defined x-rayprotocols carried out by radiology staff. In such a protocol, thepatient may be required to be accurately aligned with the x-raycassette, and the affected hip to be close to and centered with respectto the cassette. X-ray parameters, such as the fixed focal distance, maybe recorded or annotated in the resulting digital medical image (e.g.Digital Imaging and Communications in Medicine (DICOM®) format—DICOM isa registered trademark in the United States of the National ElectricalManufacturers Association). Standard format x-rays may be used to imagethe entire musculoskeletal region of interest, including the pelvis andproximal femur (and lumbar spine, if desired).

In THA, an AP Pelvis view is standard for pre-operative planning andtemplating, as well as for post-operative confirmation of implantposition. An AP Pelvis and two functional images of a patient (pre- orpost THA) may be accessed, for example, using a computer implementedmethod. The functional images may be a standing lateral and sittinglateral view of the patient's pelvic region. The images are provided fordisplay. The AP Pelvis may be a standing AP pelvis.

Spatial transformations between each image may be partly set at the timeof image acquisition. For example, in each x-ray, the cassette may bealigned with the direction of gravity (i.e. the y-axis of the resultingmedical image is parallel to the direction of gravity). Partly settingthe spatial transformations during image acquisition may decrease thenumber of steps required to determine the full spatial transformation.

With reference to FIG. 3 , there is illustrated a portion of a graphicaluser interface (GUI) 300 configured to receive input to definecoordinates from positions on the screen that are associated topositions on an anatomy shown in an x-ray. Coordinates on a standing APx-ray 302 may be identified to set at least one reference axis. In FIG.3 , coordinates of the most inferior aspect of the bilateral ischialtuberosities may be identified by a user and received by a computer(e.g. a user clicking on both points 304A and 304B using a mouse ortouch screen). In this case the reference axis 306 is denoted the“inclination reference” (though such need not be labeled on any GUI),since radiographic cup inclination may be measured relative to thisaxis. Other landmarks may be used to define this or any other referenceaxis, where the reference axis is a relevant axis to describe theposition of the implant 308.

Coordinates on the standing lateral x-ray may be identified to set atleast one reference axis. With reference to FIG. 4 , there isillustrated a portion of a graphical user interface (GUI) 400 configuredto receive input to define coordinates from positions on the screen thatare associated to positions on an anatomy shown in an x-ray 402 (astanding lateral x-ray). In FIG. 4 , coordinates of the Anterior PelvicPlane (APP) (i.e., the anterior superior iliac spine (ASIS) and pubicsymphysis) may be identified by a user and received by a computer (e.g.a user clicking on both points 404A and 404B using a mouse or touchscreen). Optionally, the angle 410 between the reference axis 406 andthe y-axis 408 (shown as a broken line) of the medical image may becomputed and displayed. In this case the reference axis is denoted the“anteversion reference”, since cup anteversion is zero when the cup faceis perpendicular to this axis. The APP is a commonly used anatomicalplane, representing the plane of the pelvis. Other landmarks may be usedto define this or any other reference axis, where the reference axis isa relevant axis to describe the position of the implant.

Coordinates on the standing lateral x-ray associated with themusculoskeletal structure (i.e. the pelvis) may be identified toestablish a standing pelvic baseline position. With reference to FIG. 5, there is illustrated a portion of a graphical user interface (GUI) 400as shown in FIG. 4 configured to receive input to define coordinatesfrom positions on the screen that are associated to positions on ananatomy shown in an x-ray 402 (a standing lateral x-ray as in FIG. 4 ).In FIG. 5 the x-ray is enlarged (zoomed in). In FIG. 5 , the center ofthe femoral head 502A and the midpoint of the sacral endplate 502Bestablish a standing pelvic baseline position. Coordinates on thesitting lateral x-ray corresponding to the same landmarks of thestanding lateral x-ray are identified to determine a sitting pelvicposition.

With reference to FIG. 6 , there is illustrated a portion of a graphicaluser interface (GUI) 600 configured to receive input to definecoordinates from positions on the screen that are associated topositions on an anatomy shown in an x-ray 602 (a sitting lateral x-ray).In FIG. 6 , the center of the femoral head 602A and the midpoint of thesacral endplate 602B are again identified. A computing unit calculatesthe change in pelvic tilt angle based on the coordinates of the standingand lateral images. This angle may be displayed, as in FIG. 6 (ΔPT 604).

A GUI may be provided such as via a display device that provides asimultaneous display of two or more x-rays of a same anatomy such asfrom different views. The GUI may be configured to receive input todefine coordinates for (e.g. over) each of the x-rays.

FIGS. 7 and 8 illustrate a GUI showing multiple images (e.g. x-rays,MRI, EOS or other modalities) of a same anatomy simultaneously where theGUI is configured to receive input to define coordinates from positionson the screen that are associated to positions on an anatomy, amongother features of the GUI.

More particularly, FIG. 7 shows a GUI 700 to present three x-rays 302,402 and 602 simultaneously comprising respectively a standing AP view, astanding lateral view and a sitting lateral view. GUI 700 shows controls702 along a top edge, for example, to zoom in or out, adjust displayproperties such as contrast and brightness, adjust or scroll the imagein the display, input various coordinates, choose between left and righthip (anatomy) and choose between radiographic and operative definitionsettings.

The computing unit determines the spatial transformations between allthree images. In instances where the image acquisition method providesknown information about the spatial transformations, this knowninformation may be used in the calculation. Examples of knowninformation include any combination of the following:

-   -   AP view is perpendicular to lateral view    -   AP and or lateral views are aligned to coronal and/or sagittal        anatomical planes    -   The direction of gravity projects onto the y-axis of the        resulting images    -   The projection and/or distortion models of the imaging equipment

Computing the spatial transformations may utilize the art of imageregistration, in which corresponding common features across multipleimages are identified, and provided to a solving algorithm, where thesolving algorithm determines spatial transformations based on 3Drotations and translations. For example, a spatial transformation may becomputed between an AP and lateral pelvis x-ray by identifying at least3 pairs of corresponding landmarks on each image (such as one ASIS, thepubis, and the inferior point of the sacrum). A coordinate frame may begenerated based on the landmarks in each image, and the transformationbetween the two coordinate frames may be computed as the spatialtransformation between both images. In another example, a vector commonto two images may be used in the computation of the spatialtransformation.

Computing spatial transformations may include a combination of any ofthe methods described herein as well.

Image registration may utilize 3D information about the anatomicalstructure of interest (e.g. the pelvis). The 3D information about theanatomical structure may be based on a patient scan (e.g. a segmentedpelvis derived from a CT scan). The 3D information may be based on ageneric patient template, optionally modified for a patient's parameterssuch as: height, weight, BMI, sex, race, etc.). The 3D information maybe used as a constraint in an optimization computation that determinesthe spatial transformation between two images.

The computing unit may access the 3D model of an implant (e.g. anacetabular cup). The implant model may be generic (i.e. such as ahemisphere representing an acetabular cup), or specific to a design of acommercially available acetabular cup implant (e.g. based off of CAD).The 3D model may be rendered and displayed on each of the three images,as shown in FIG. 7 . The implant is rendered for each image according tothe reference axes and/or spatial transformations and/or the change infunctional position, based on a consistent 3D position relative to themusculoskeletal structure (e.g. the pelvis) in each image. The 3D modelmay be initially rendered and displayed in a first, or default,position. For THA, the default may be for the cup to be at 40° ofinclination and 15° of anteversion (704 and 706 as in FIG. 7 ), which isthe center of a clinically accepted safe zone (See e.g. Lewinnek et al.Dislocations after total hip replacement surgeries, The Journal of Jointand Bone Surgery (JBJS) 1978; Kanawade et al. Predictability ofAcetabular Component Angular Change with Postural Shift from Standing toSitting Position, JBJS 2014; Buckland et al. Acetabular AnteversionChanges Due to Spinal Deformity Correction: Bridging the Gap Between Hipand Spine Surgeons, JBJS 2015).

Alternatively, the default cup angle may result from a computation tominimize impingement and/or dislocation risk (or maximize jointstability through functional patient movements). For example, acomputation to minimize impingement and/or dislocation risk may receivethe change in functional position data (i.e. the change in pelvic tiltangle), optionally receive other data, such as the desired hip range ofmotion, and compute a cup angle to be used as the default. Thecomputation may involve computer-implemented simulation, or a look-uptable based on clinical research correlating dislocation and/orimpingement risk with change in functional cup position. A default cupposition may be chosen by shifting cup position within a known safezone. The cup position may be shifted within the safe zone based on oneor more patient-specific attributes. The shift may be based on thecorrelation of the tile and anteversion. As described further hereinbelow with reference to FIGS. 24-26, 27A and 27B, a safe zonevisualization (or more than one) may be rendered and displayed tovisually indicate the location of the safe zone (or safe zones) withrespect to one or more of the 3 images. Alternatively, or additionally,placement may be based on a patient's sex, height, weight, Body MassIndex, or more.

The computing unit may be responsive to real time changes to theposition of the cup (i.e. changes from the default position). Forexample, a user may drag one or more handles (using a mouse, trackpad,touchscreen, or other input device for the computing unit) associatedwith the 3D implant rendering to change its position. In another manner,controls relative to the measurements may receive input (e.g. numericinput) to change the value of the measurements (anteversion andinclination) to instruct the GUI to position the cup implant with suchmeasurements.

FIG. 8 is an illustration of the GUI 700 showing a cup implant 308 witha changed position relative to its position in FIG. 7 . Changing theposition of a cup implant 308 using a control with respect to one of thex-rays automatically updates in all views to maintain a consistent 3Dposition relative to the musculoskeletal structure. The change inposition for an acetabular cup includes the angles of inclination andanteversion. The change in position may optionally include thetranslational position and/or the size of the rendered implant.Alternatively, the size and translational position may be independentlyadjusted for each of the three images. Changes made to the position ofthe implant would preferably be responsive to user input in real time,so that the user doesn't experience frustrating and / or potentiallyconfusing lag.

In FIG. 7 and FIG. 8 there are fields labeled Inclination andAnteversion, which may update in response to a user modifying theposition of the rendered implant, and/or may be user-modifiable suchthat a user could enter a particular cup angle, and the rendering wouldupdate accordingly.

In FIG. 7 and FIG. 8 , controls 702 illustrate a radio button control800 to toggle between radiographic and operative definitions of cupangles. The computer implemented method would calculate and display theInclination and Anteversion measurements according to the respective cupangle definitions (in mathematical terms, this amounts to using adifferent set of Euler angles to describe a given 3D rotation).

The use of two standing x-ray views from orthogonal perspectives hasbeen described, and is advantageous for simplifying the process ofdetermining the spatial transformations (since the views are orthogonal,and have a common gravity vector (parallel to the vertical image marginin both images) with respect to the anatomical structures). Two x-rayviews from different perspectives may be used without the constraint ofsharing a common vector, such as gravity. For example, in THA, often APpelvis x-rays are taken with the patient supine. Likewise, CT scanstypically having the patient lying supine during image acquisition. Thespatial transformation between the images of different perspectives maybe calculated without relying on a common direction of gravity (i.e. asdescribed previously wherein image registration techniques are used todetermine the spatial transformation). A computer-implemented method mayprovide alternative methods for calculating the spatial transformations,where one alternative relies on two different views sharing a commonvector (e.g. gravity), and the other alternative does not, but reliesinstead on matching anatomical landmarks via image registration. Thecomputer-implemented method may receive user input to indicate which ofthe two alternative methods to use for a given set of images.

The use of more than three images is contemplated. More than two imagesrelated by a functional motion may be used. For example, in addition toa sitting and standing lateral image of a pelvis, a flexed-sitting viewand/or a step-up view (in which a patient has their contra-lateral legraised as if they are “stepping up”) may be provided. Thecomputer-implemented method may receive the additional functional views,compute the change in functional position parameters (e.g. change inpelvic tilt), compute the respective spatial relationships, and renderand display a 3D implant accordingly on each view. The use of additionalnon-functional views is also contemplated. For example, a supine AP hipx-ray is commonly used in clinical practice. The spatial transformationbetween this view and the other views may be computed, and the 3Dimplant may be rendered on this image based on the spatialtransformations. In another example, spatial transformation betweenfunctional images and a CT scan enables 3D implant overlays on a supinecoronal, supine transverse and supine sagittal view.

The computer-implemented method described herein may utilize any 3Dimplant model, corresponding to the implants of a particular surgery.For example, in a THA, the 3D implant model of a femoral prosthesis(either a generic model, or a manufacturer-specific model) may be usedinstead of, or in addition to an acetabular cup 3D implant model. In thecase of a femoral prosthesis, the musculoskeletal structure associatedwith this implant is a femur, and reference axes associated with thefemur may be identified in the respective medical images. As a result,renderings of both the femur stem and acetabular cup implants may beprovided on the x-ray views (e.g. the AP pelvis, the standing lateralx-ray and the sitting lateral x-ray). The version of the femur stem isan important clinical parameter, since it is controllable by the surgeonduring a THA (whereas flexion and/or varus angles of the femur stem aregenerally enforced by the anatomy). The version of the femur stem may bea parameter representing the position of the femur stem (analogous toInclination and Anteversion for an acetabular cup implant). The positionof the femur stem may be updated in real time by invoking a processorexecuting computer instructions via clicking on handles associated withthe 3D rendering and/or manually entering values into a field for femurstem position. As with the previously-described acetabular implantposition, the position of the femur stem rendering may update in realtime on each image based on a consistent 3D position relative to thefemur.

In THA, digital templating (aka implant sizing) is commonly performed topredict implant sizes, and determine an expected position of theprostheses. Digital templating is typically performed on an AP pelvis orAP hip x-ray. The computer-implemented method herein may implement atemplating module, so that a surgeon has a single software interface toperform their pre-operative planning. Alternatively, thecomputer-implemented method may receive templating information toinfluence how or where the implant is rendered on the image. Forexample, the templating information may include: implant size, implantmake/model, image scaling factor, implant translational position. Whenrendering the 3D implant on the respective images, they may be renderedto achieve consistency with the templating information (e.g. the centerand size of an acetabular prosthesis as templated causes the 3D modelrendering to be centered at the same location in the image, with thesame size).

The computer-implemented method described herein may be used forpre-surgical planning, or post-surgical validation, or both. Where bothpre-operative and post-surgical images are used, changes between thepre- and post-references axes and change in tilt parameters may becalculated and provided for display to a user (for example, to quantifyhow the change in pelvic tilt during functional motions changed as aresult of the surgical procedure).

The computer-implemented method may be implemented using a computingdevice (sometimes referenced as a computing unit). Representative butnon-limiting examples include a laptop, PC, workstation, server, tabletand smartphone.

FIG. 9 is a diagram illustrating in block form an example computingdevice (e.g. 900), in accordance with one or more aspects of the presentdisclosure, for example, to provide a computing device to perform to anyone of the method aspects described herein. Computing device 900comprises one or more processors 902, one or more input devices 904, adisplay device 906, one or more communication units 908 and one or moreoutput devices 910. Computing device 900 also includes one or morestorage devices 912 storing one or more modules (e.g. as software) suchas medical image module 914, implant model module 916, display module218 and communication module 920. Communication channels (e.g. 922) maycouple each of the components 902-920 for inter-componentcommunications, whether communicatively, physically and/or operatively.In some examples, communication channels 922 may include a system bus, anetwork connection, an inter-process communication data structure, orany other method for communicating data.

One or more processors 902 may implement functionality and/or executeinstructions within computing device 900. For example, processors 902may be configured to receive instructions and/or data from storagedevices 912 to execute the functionality of the modules shown in FIG. 2, among others (e.g. operating system, other applications, etc.)Computing device 900 may store data/information to storage devices 912.Computing device 900 may be coupled to external storage devices (notshown), whether they are located closely or remotely from device 900.For example, remote storage devices (not shown) may be accessible via aserver or other computing device to receive images of a patient. Theimages may be copied and stored locally in storage devices 912 or inexternal devices, such as for use during preoperative planning orpostoperative review.

One or more communication units 908 may communicate with externaldevices such as servers (not shown), etc. via one or more networks (notshown) by transmitting and/or receiving network signals on the one ormore networks. The communication units may include various antennaeand/or network interface cards, etc. for wireless and/or wiredcommunications.

Input and output devices may include any of one or more buttons,switches, pointing devices, cameras, a keyboard, a microphone, one ormore sensors (e.g. biometric, etc.) a speaker, a bell, one or morelights, etc. One or more of same may be coupled via a universal serialbus (USB) or other communication channel (e.g. 922). In the presentexample, computing device 900 comprises a display device 906. In otherexamples, computing device is coupled to an external display device (notshown). Of course device 900 may have a display device 906 and also becoupled to an external display device. The display device may beconfigured to provide input to device 900 such as via a touch screen,etc.

The one or more storage devices 912 may store instructions and/or datafor processing during operation of computing device 900. The one or morestorage devices may take different forms and/or configurations, forexample, as short-term memory or long-term memory. Storage devices 912may be configured for short-term storage of information as volatilememory, which does not retain stored contents when power is removed.Volatile memory examples include random access memory (RAM), dynamicrandom access memory (DRAM), static random access memory (SRAM), etc.Storage devices 912, in some examples, also include one or morecomputer-readable storage media, for example, to store larger amounts ofinformation than volatile memory and/or to store such information forlong term, retaining information when power is removed. Non-volatilememory examples include magnetic hard discs, optical discs, floppydiscs, flash memories, or forms of electrically programmable memory(EPROM) or electrically erasable and programmable (EEPROM) memory.

Medical image module 914 may be configured to receive and work with thevarious forms of medical images as described herein. For example, it maybe configured to determine various axis and transformations, etc. andprepare image data therefrom for display via display module 918. Implantmodel module 916 may be configured to receive implant model data andwork with it to enable the renderings and movement/manipulation of therenderings via input from the one or more of the input devices asdescribed herein. Display module 918 may display images from modules 914and 916. It is understood that operations may not fall exactly withinthe modules 914-920 of FIG. 9 such that one module may assist with thefunctionality of another. Additional modules may be stored (not shown).

FIG. 10 is a flowchart showing operations 1000 of a computer implementedmethod, such as may be implemented by computing device 900. At 1002operations access and display multiple images showing anatomy forexample, a standing AP image, a standing lateral image, and a sittinglateral image of a musculoskeletal structure of a patient. At 1004operations receive respective axes coordinates on two x-rays fromdifferent points of view (e.g. the standing AP image and standinglateral image respectively) as input, and define reference axes of themusculoskeletal structure on the respective x-rays (e.g. standing APimage and the standing lateral image) based on the respective axescoordinates.

At 1006 operations receive respective tilt coordinates (e.g. on thestanding lateral image and the sitting lateral image respectively) asinput, and determine a change in tilt parameter (between the standing APimage and the standing lateral image) of the musculoskeletal structurebased on the respective tilt coordinates.

At 1008 operations determine relevant respective spatial transformationsbetween pairs of x-rays (e.g. between pairs of images in differentpoints of view and pairs of images in a same point of view but showingdifferent positions of the anatomy) For example, operations maydetermine a spatial transformation between the standing AP image and thestanding lateral image and a spatial transformation between the standinglateral image and the sitting lateral image.

Operations render and overlay a 3D implant for each of the x-rays usingthe reference axes, spatial transformations and change in tiltparameter. For example, at 1010 operations render and overlay a 3Dimplant for the standing AP image and the standing lateral image in afirst position relative to the reference axes based on the referenceaxes and the spatial transformation between the standing AP image andthe standing lateral image. At 1012 operations render and overlay a 3Dimplant in the first position for the sitting lateral image based on thechange in tilt parameter and the spatial transformation between thestanding lateral image and the sitting lateral image.

Operations may receive input to move the implant to a second positionand update a rendering and overlay of the implant to the second positionin each of the x-rays, e.g. in real time. At 1014 operations receiveinput representing a second position of the 3D implant; and, at 1016,render and overlay the 3D implant in the second position using eachspace transformation and the change in tilt parameter to update, in realtime, each of the standing AP image, the standing lateral image and thesitting lateral image.

It is understood that the images may be from any of the followingmodalities: x-ray, CT, MRI, EOS. The computer implemented method maydisplay the change in tilt parameter (and any of the reference axes orother measurements). Determining spatial transformations may compriseone or more of: performing computations based at least in part on thereceived coordinates; performing computations based on correspondingcommon features between received images; and performing computationsbased on known image acquisition spatial information.

FIG. 11 is a flowchart of operations 1100 of a computing device such ascomputing device 900. At 1102 operations access and display at leastthree images of a musculoskeletal structure of a patient, where one pairof images are of different views of a first position, and another pairof images are of the same view in different functional positions. At1104 operations define reference axes of the musculoskeletal structureon at least two images, based on user input. At 1106 operationsdetermine a positional change parameter of the musculoskeletal structurebased on the two images of a common view in different functionalpositions, the positional change parameter representing the positionalchange of the patient between the functional positions. At 1108operations determining spatial transformations between the images. At1110 operations render and overlay a 3D implant for each image in afirst position relative to the reference axes, and according to therespective spatial transformations, on the respective images. At 1112operations receive input representing a second position, and, at 1114,update, in at least near real time, the render and overlay accordinglyfor each image.

Co-Registered Image Views

Multiple combinations of medical images can be co-registered within thecomputing system (i.e. in accordance with the computer implementedmethod) to provide information about the relationships of the imagecoordinate systems for the purpose of cup visualization across thecoordinate systems and relative to the different views. These medicalimages can include Standing AP, Standing Lateral, Sitting Lateral andSupine AP image views.

Standing AP to Standing Lateral

The Standing AP and Standing Lateral images may be co-registered bydesign/assumption. The computer implemented method assumes that thevertical axis of both images are parallel, and the horizontal axes ofthe images are 90 degrees offset from each other. This may be achievedwith EOS imaging system (https://www.eos-imaging.com/), or withconventional radiographs and careful patient positioning during imageacquisition. A patient may be directed to rotate 90°, pelvic landmarksmay be used, a positioning jig may be used, etc.).

FIGS. 12A and 12B are EOS images 1200, 1202 of an anatomy from two viewsshowing reference axes in each view. In both EOS images 1200, 1202, thevertical axes 1204, 1206 are parallel. The horizontal axis 1208 of theAP image (EOS image 1200) is assumed to be perpendicular to the lateralimage plane, and the horizontal axis 1210 of the lateral image (EOSimage 1202) is assumed to be perpendicular to the AP image plane.

Standing Lateral to Sitting Lateral

To co-register the standing lateral to sitting images, the computerimplemented method assumes that the pelvises in both lateral images haveonly rotated in the image plane, and can be measured by identicalmeasuring pelvic landmarks on both images. One such landmark that couldbe used is the sacral slope (angle of the superior aspect of thesacrum).

FIG. 13 is a portion of a GUI 1300 showing two images 1301 and 1302.Input has been received to mark coordinates to indicate respectiveslopes. The shorter lines 1304 and 1306 between the pairs of roundhandles (e.g. 1304A, 1304B and 1306A, 1306B) are used to measure thesacral slope of the pelvis in the two images 1300 and 1302. Thedifference in the angle of the sacral slope is −50 degrees (50 degreesof posterior rotation) and is displayed (1308).

Other landmarks can be used (e.g. hip center to operative ASIS, or pubisto ASIS) as long as the same reference line is used on both images.

Standing AP to Supine AP

To co-register the two AP images, operations of the computer implementedmethod calculate to predict a change in Pelvic Tilt between two APradiographs such as by crafting and using a model. On each image severalbony landmarks are measured—a distance from the top of the pubis to thetrans-ischial line, and a pelvic outlet diameter. 3D models, andsynthetic x-rays with known pelvic tilts, were employed to use thesepelvic landmarks to model change in pelvic tilt. Thus a look up table(e.g. a nomogram) was defined using representative population data.

The model is a linear model which calculates pelvic tilt for one image(pelvic tilt relative to the image plane), and the difference in pelvictilt is reported:

${pt} = {{\left( \frac{PSTI}{POD} \right)*m} + b}$

Where PSTI is the line from the top of the pubis (PUBIS) to the transischial line (TI), POD is the pelvic outlet diameter, and m and b arethe coefficients of the model, which are gender specific.

One method of determining this model is to collect both known (i.e.measured) model inputs (e.g. PSTI and POD) and known (i.e. measured)model outputs (e.g. PT) from a representative set of sample data (e.g. astatistically significant quantity of radiographs with representativedemographics (gender, race, age, BMI)). With a paired list of knowninputs and outputs to a model, it is then possible to determine themodel parameters (e.g. m and b) using any linear or non-linearoptimization method. The example described here is a linear model withtwo inputs (PSTI and POD) and one output (PT), but the model may also benon-linear with fewer or greater numbers of inputs.

In accordance with the method, the difference in pelvic tilt (PTD) (e.g.19°) may be displayed, such as via a GUI.

With reference to FIG. 14 there is shown a portion of a GUI 1400 showingtwo images 1401 and 1402, where on both images, the length of the line1404 from the top of the pubis to the trans-ischial line 1406 ismeasured and scaled by the pelvic outlet diameter (line 1408). Thisratio is plugged into a model developed to measure pelvic tilt. Thisexample shows (at GUI element 1410) a pelvic tilt of 19 degrees in theanterior direction.

Other landmarks may be used, for example, the “teardrop” line (notillustrated with a line in FIG. 14 ) may be used instead of thetrans-ischial line and a model developed.

FIG. 15 is a flowchart of operations 1500 for a computing device such ascomputing device 900.

At 1502 operations access and display a first AP image and a second APimage of a musculoskeletal structure of a patient including a pelvis,where the first and second images are of the same view in differentfunctional positions. At 1504 operations receive coordinates ofspecified landmarks on the first AP image as input. At 1506 operationsreceive coordinates of the specified landmarks on the second AP image asinput. At 1508 operations compute a change in AP tilt parameter based onthe received coordinates from the first and second AP images using acomputer function that implements a look up table of coefficientsmapping coordinates to tilt parameters, wherein the look up table isgenerated using representative population data. At 1510 operationsco-registering the first AP image and second AP image using the changein AP tilt parameter to overlay an object in a same position over eachof the first AP image and second AP image in a graphical user interface.The change in AP tilt parameter may be provided for display (1512).

The computer implemented method may further comprise rendering a 3Dimplant (an example of an object) and overlaying it for each of thefirst AP image and the second AP image in a first position based onreference axes and the change in tilt parameter. The computerimplemented method may further comprise displaying numerical valuesindicating the first position (e.g. angles relative to the referenceaxes).

The 3D implant may be an acetabular cup and the musculoskeletalstructure may be a pelvis of a patient. The numerical values may beinclination and anteversion, and wherein inclination is relative to amedial-lateral reference axis, and anteversion is relative to the planeof the x-ray for each image. In this computer implemented method, thefirst AP image may be a standing AP Pelvis x-ray, and the second APimage may be a supine Pelvis x-ray. The specified landmarks of thepelvis are the inferior bilateral ischial tuberosities, the pubicsymphysis, and the lateral-most points along the pelvic brim.

Computation of the of the AP tilt parameter may be based on populationdata that is different for males and females, and the method may includereceiving input to indicate the gender of the patient.

The computer implemented method regarding the AP tilt parameter may beperformed in conjunction with other computer implemented methodsdescribed herein. For example it may be performed following a computerimplemented method to determine spatial transformations between pairs ofimages (e.g. a pair of images comprising different views of a sameposition or a pair of images comprising same views of differentfunctional positions) using references axes determined for such imagesand rendering and displaying a 3D implant over the images using thespatial transformation(s) as applicable.

Functional to APP

FIGS. 16A-16C are illustrations of portions of a GUI 1600A, 1600B and1600C showing an image 1602 and enabled to receive input of coordinatesand determine (e.g. compute) clinically relevant values therefrom, amongother features. Each of FIGS. 16A-16C show a standing AP image (image1602) where FIG. 16A shows a larger portion thereof and FIGS. 16B-16Cshow a smaller portion thereof for ease of illustration.

In accordance with a computer implemented method, to calculate theregistration between the functional position (standing coronal), and theAPP (a commonly accepted pelvic reference frame) there is measured ahorizontal reference 1604A, 1604B and 1604C (inclination reference) inthe standing AP image, and a vertical reference (anteversion reference)in the standing lateral image (See FIG. 17 ).

The horizontal reference can be any number of clinically acceptedlandmarks. It can be the inter-ischial line, the teardrop line, the linebetween the bottom of the obturator foramens, or any other set oflandmarks the user deems appropriate for a horizontal reference.Examples of the different horizontal references (1604A, 1604B and 1604C)are shown in FIGS. 16A-16C where a Trans/Inter Ischial line is shown inFIG. 16A (left), a bottom of obturator foramens is shown in FIG. 16B(top right), and a teardrop line in FIG. 16C (bottom right).

With reference to FIG. 17 showing a portion of a GUI 1700 showing animage 1702 comprising a standing lateral image, the vertical reference1704 is always the line from the pubic symphysis to the ASIS points. Inthe case where the ASIS points are not superimposed, the line can bisectthe two ASIS points. As illustrated in FIG. 17 , the vertical referenceline (VR) is shown from the pubis (bottom white circle), and bisectingthe two ASIS points (top white circle). The measured tilt of the pelvis,relative to the standing coronal plane, is −10.3 degrees (posteriorpelvic tilt) as shown in the graphical element 1706.

Cup Visualization on Co-Registered Views

When all views have been co-registered, a visual representation of a cupimplant can be simultaneously viewed on any or all images, and reportedin all measurement planes.

FIG. 18 is an illustration of a portion of a GUI 1800 having similarfunctions and features of the other GUIs and comprising an instancewhere all four images 1802, 1804, 1806 and 1808 are present and a cupoverlay 1810 (CI) is visualized. In this case there are visualizationson all four images, and they show what the shape of the cup as if therehad been a cup present during the imaging. There are also measurements1812A and 1812B reported in various clinically relevant coordinatesystems, including the standing coronal (1812A), and supine coronalreference frames (1812B), as well as the APP reference frame (1812A).

These cup visualizations 1810A, 1810B, 1810C and 1810D and reportedangular measurements (1812A, 1812B) are linked. Changing any one of thevisualizations (i.e. a position of one of the cup implants 1810A, 1810B,1810C and 1810D over any one of the images 1802, 1804, 1806 or 1808)adjusts all the other visualizations, as well as the reportedmeasurements. If the visualization 1810C of the cup implant is changed,then visualizations 1810A, 1810B and 1810D are changed automatically.

FIG. 19 illustrates an enlarged portion of the GUI 1800 showing cupimplant 1810A and in which the input has been received to select orinvoke a control therefor (e.g. by selecting the cup implant in the GUI1800) to change the position of the cup implant. In accordance with thecomputer implemented method the GUI may be configured so that the cupoverlay CI in any of the image views can be adjusted (moved) usinghandles on the respective objects (FIG. 19 ). Using the handles in oneimage moves the CI accordingly in all images, preferably in near realtime to avoid any lag or negative user experience. The cup implant canbe adjusted in size (bottom floating handle H1 1902), anteversion(middle handle H2 1904), or inclination (top handle H3 1906).

Alternatively or in addition, the cup overlay in the images may beadjusted (moved) by inputting new values into the measurement text boxesto change an angle. The CI position is then adjusted in all images usingthe image co-registrations.

Multiple Displays of Cup Angles in Different Reference Frames

All the visualizations and measurements displayed (e.g. in FIG. 18 )represent a single cup CI placed at a given orientation in the pelvis,and represented in various coordinate systems using the co-registrationsdescribed above.

The inclination and anteversion angular measurements can be reportedusing two common definitions, Radiographic, and Operative, using atoggle switch 1814 (in the bottom left). Changing the anglerepresentation does not affect the visual representation. That is,toggling the definition changes the text comprising the specificmeasurements shown on the screen, but not the visualization. Input tochange the specific measurements (in either definition) updates thevisualization according to medically accepted relationships.

In the different image panes 1816A, 1816B, 1816C and 1816D, the cup maybe visualized as a projection onto the image plane to help the uservisually align the cup. The user also uses cup angles to determineproper cup positioning. These angles can be calculated according tomultiple reference frames, such as the standing coronal, supine coronal,and anterior pelvic planes. These measurements can all be displayedsimultaneously as shown in measurement display text boxes in FIG. 18 .

Ante-Inclination Lateral

Ante-inclination (AI 1820) is the angle the major axis of a cupprojected onto a lateral image makes with the horizontal margin of thatimage. The measurement of AI 1820 is invariant to the selecteddefinition of angular measurements and is only determined using thecurrent cup visualization position. The measurement can be seen in FIG.18 as graphical element 1822. The ante-inclination in various positionshas been shown to correlate with various clinically relevantconsiderations, such as dislocations, or the stiffness of the spine.

FIG. 20 is a flowchart of operations 2000 for a computing device such ascomputing device 900. The operations may provide a method to render andoverlay a 3D implant in 3 images simultaneously. The method may beperformed after determining a change in AP tilt such as previouslydescribed with reference to FIG. 15 , or in another manner.

At 2002, operations access and display a first image, a second image anda third image of the musculoskeletal structure of the patient, where thefirst image and the third image are images are of different views of afirst patient position and together the first image, the second imageand the third image define three images. At 2004, operations definereference axes of the musculoskeletal structure on the three images,based on user input. At 2006 operations determine a spatialtransformation between the first image and the third image based on thereferences axes. At 2008 operations render and overlay a 3D implant in afirst implant position on each of the three images responsive to thereference axes, where overlaying on the second image is furtherresponsive to the change in AP tilt and where overlay on the third imageis further responsive to the spatial transformation.

The operations 2000 may comprise receiving input to overlay the 3Dimplant in a second implant position and updating the overlaying of the3D implant in the second implant position in the three images responsiverespectively, to the change in AP tilt and the spatial transformation.

FIG. 21 is a flowchart of operations 2100 for a computing device such ascomputing device 900. Operations may provide a method to simultaneouslydisplay at least three images of musculoskeletal structure associatedwith different planes, for example, a Standing Coronal Plane, a SupineCoronal Plane, and an Anterior Pelvic Plane.

At 2102 operations access and display in a UI at least three images of amusculoskeletal structure of a patient, the musculoskeletal structurebeing associated with different planes including a Standing CoronalPlane, a Supine Coronal Plane, and an Anterior Pelvic Plane. At 2104operations co-register the at least three images responsive to therespective planes and to reference axes defined therefore in response toinput received via the UI to generate respective spatial transformationsbetween pairs of the at least three images. At 2106 operations renderand overlay on the at least three images a 3D implant in a first implantposition defined by inclination and anteversion measures relative to oneof the different planes and wherein the overlaying is further responsiveto the reference axes and the respective spatial transformations. At2108 operations determine equivalent inclination and anteversionmeasures relative to another of the different planes. At 2110 operationsdisplay the inclination and anteversion measures and the equivalentinclination and anteversion measures in the UI simultaneously and inreal time.

Operations 2100 of the computer implemented method may further comprise:receiving an input via the UI to move the 3D implant to a second implantposition defined by inclination and anteversion measures relative to oneof the different planes; rendering and overlaying the 3D implant in theat least three images in accordance with the second implant position,the overlaying further responsive to the reference axes and therespective spatial transformations; determining equivalent inclinationand anteversion measures for the second implant position relative toanother of the different planes; and displaying the inclination andanteversion measures and the equivalent inclination and anteversionmeasures for the second implant position in the UI simultaneously and inreal time.

FIG. 22 is a flowchart of operations 2200 for a computing device such ascomputing device 900. Operations 2200 provide a computer implementmethod to determine (and optionally display) an ante-inclinationmeasure.

At 2200 operations access and display in a UI at least two images of amusculoskeletal structure of a patient, the at least two imagescomprising one or more lateral views. At 2204 operations co-register theat least two images responsive to respective planes and to referenceaxes defined therefore in response to input received via the UI togenerate respective spatial transformations between pairs of the atleast two images. At 2206 operations render and overlay on the at leasttwo images a 3D implant in a first implant position, the overlayingresponsive to the reference axes and the respective spatialtransformations. At 2208 operations determine and display each of theone or more lateral views a respective ante-inclination measure for the3D implant.

Operations 2200 may further comprise: receiving an input via the UI tomove the 3D implant to a second implant position; rendering andoverlaying the 3D implant in the at least two images in accordance withthe second implant position, the overlaying further responsive to thereference axes and the respective spatial transformations; anddetermining and displaying in each of the one or more lateral views arespective ante-inclination measure for the second implant position.

Computer Tablet Integration

Any of the computer methods herein may be configured to work through aweb browser on a tablet. The required imaging can be uploaded to thetablet such as by using a camera on the tablet. A picture of a screendisplaying the respective radiographic images may be taken andidentified. Since it is difficult to ensure that the tablet is perfectlypositioned, it is possible to correct for rotation and skew error asshown in FIG. 23 .

One method of accomplishing this correction involves the identificationof the four image corners, either through manual selection, throughautomatic image processing algorithms, or a combination of both, andcalculating a transformation which causes these corners to be squarewith the edges of the image parallel to the edges of the display,applying this transformation to some or all points on the image. Aperson of ordinary skill will be aware of unwarping and similar suchtechniques.

Safe Zone Visualization

As briefly described herein above, surgeons often reference a so-called“safe zone” of the cup. This refers to a region (defining a 3D space) ofcup angles which could be a factor in reducing post-operativedislocations. The safe zone is generally described by aninclination/anteversion pair of angles, and a range of acceptabledeviation. One such safe zone might be 40/20 degrees radiographicinclination/anteversion and ±10 degrees of range. In accordance with theteachings of various practitioners, etc., different safe zones and/orranges may be proposed.

A computer implemented method may provide a visualization such as a safezone graphical element rendered and overlaid for one safe zone or morethan one safe zone. FIGS. 24-26 are illustrations of a portion of a GUI2400 showing a save zone visualization over images 2402, 2404 and 2406.FIG. 26 shows only images 2402 and 2404. In FIGS. 24-26 , the safe zone(SZ) 2408A, 2408B and 2408C is visualized by a safe zone graphicalelement in the form of an arc which may be colored on a display screen,with a dotted line (e.g. a central cup axis (CA) 2410A, 2410B and 2410C)perpendicular to the cup face indicating the position of the cup isinside or outside this arc. Color is not shown in the drawings as filed.A dotted line CP 2412A, 2412B and 2412C may also illustrate the cupplane across the cup face. In FIG. 24 , GUI 2400 shows StandingAP/Standing Lateral cups are in the safe zone and Sitting Lateral cup isoutside of the safe zone. FIG. 25 shows SZ 2408C as opaque when thesitting lateral cup is outside the safe zone.

The safe zone graphical element may be differently displayed in each ofthe images. For each of the images the respective safe zone graphicalelement may have respective graphical characteristics. At least one ofthese characteristics is selected according to whether (relative to aparticular image) the 3D implant is in the safe zone in the particularimage. For example, a characteristic may be color. A color of the arcmay be used to indicate when the cup is within the safe zone (forexample, green for inside, red for outside of the safe zone). In FIG. 24, the safe zone of the Sitting Lateral view (relative to images 2406) isred when viewed in color and highlighted with a dotted white outline foremphasis to show the cup is outside (without) the safe zone SZ 2408C.Other visualization features (graphical characteristics) of the safezone may be used such as a change of state (blinking or light intensity,color changing) to distinguish the safe zone arc or the border thereoffrom inside to outside indicating.

A different overlay style may be used to distinguish in from out. Adifferent overlay style may be used such as using at least partiallytransparent color for the arc when the 3D implant is inside and usingopaque when it is outside (e.g. FIG. 25 Sitting Lateral view(rightmost)), or from a using consistent transparency to alternatingtransparent/opaque stripes or sections (e.g. warning bars, checkeredflag (not shown)), etc.

Alternatively or in addition, an audible signal or haptic feedback (e.g.device vibration) may be provided via output devices. Alternatively orin addition, the visualization of the line normal to the cup face may bevaried to indicate whether the cup is within or without of the safe zone(e.g. color, blinking, dotted vs. solid, etc. (not shown)).

The computer method may render and overlay a 3D implant respectively foreach of the at least two images in a first position relative toreference axes, and according to each respective spatial transformation;and render and overlay respectively for each of the three images a safezone graphical element indicating a clinically accepted safe range ofpositions for the 3D implant.

The safe zone graphical element may be a respective graphical elementfor each of the images. Each respective graphical element may haverespective graphical characteristics, for a particular image of theimages, the respective graphical characteristics are selected based on adetermination whether the 3D implant is positioned inside or outside theclinically accepted safe range of positions in the particular image.

As a planning position of the cup is adjusted (e.g. using object handlesin a GUI, or through the measurement input fields), the color and/orother visual features of the safe zone and/or cup axis line CA maychange and other feedback signals provided based on the angles of thecup. This gives the user intuitive visual (and/or auditory, etc.)feedback on the cup location, rather than thinking about numbers andranges. Thus, the computer method may receive input representing asecond position of the 3D implant; render and overlay accordingly the 3Dimplant in the second position to update, in at least near real time,each of the at least two images respectively; determine for eachparticular image, whether the second position is inside or outside theclinically accepted safe range of positions in the particular image;select the respective graphical characteristics accordingly; and renderand overlay the respective graphical element for each of the images.

The safe zone here is calculated relative to the horizontal reference(HR) 2414 line shown in purple in FIG. 24-26 when viewed in color) onthe standing AP image (leftmost), and relative to the coronal plane ofthe pelvis (line VR for the vertical axis of the standing lateral image(middle)). In the computer method, the clinically accepted safe range ofpositions may defined in accordance with at least two angles relative tothe reference axes and an associated range of position for each angle.

The safe zone can be imagined as a 3D sector of a sphere (e.g. definedrelative to the angles and ranges of inclination and anteversion). Thissector may be defined as a rectangular pyramidal sector shown in FIG.27A, where the sector is defined with two pairs of angles, start andend, from the horizontal equator of the sphere, and from a verticalreference line. The sector may be defined as a conical sector, wherethere is a pair of angles, relative to vertical and horizontalreferences on the sphere, and a radial distance away from this indicatedset of angles, shown in FIG. 27B.

The visualizations of that sector can be rendered in a desired style orformat as the projections on to the 2D views. As noted, three such viewsare shown in FIGS. 24 and 25 and two in FIG. 26 . A determination may bemade whether the 3D implant is inside or outside the safe zone todetermine the appropriate style or format (graphical characteristics) ofthe safe zone graphical element and/or axis line CA to render anddisplay. The dotted line from the cup, representing an axis normal tothe cup face, is present within or without the respective sector in eachparticular 2D view as a visual aid. The computer implemented method mayrender and overlay an axis line extending from the 3D implant to assistwith visualizing the 3D implant relative to the clinically accepted saferange of positions.

Particular safe zones (i.e. 3D sectors) could be defined by respectiveclinically relevant research (e.g. Lewinnek et al.; etc. as notedabove), or could be defined individually by the surgeon. A button orother GUI interface (not shown) may be provided to select a desired safezone sector determined by the respective research for visualization orto permit definition of a zone by the surgeon or other user.Alternatively or in addition (not shown) more than one of the safe zonesmay be visualized (e.g. showing safe zones from two clinically relevantresearch sources). Any overlap in the e.g. two zones may be visualizedby a third color (e.g. a first safe zone visualized in yellow, a secondsafe zone visualized in blue and any overlap visualized in green (e.g.via additive coloring)). Thus in the computer implemented method theclinically accepted safe range of positions is predefined in accordancewith a defined standard or selectively defined in accordance with inputreceived. The computer implemented method may comprise receiving inputto select between the clinically accepted safe range of positions aspredefined in accordance with the defined standard or as selectivelydefined in accordance with input received.

FIG. 28 is a flowchart of operations 2800 of a computing device such asa computing device 900. The operations may provide a computerimplemented method to visualize a safe zone relative to an implant inmultiple images.

At 2802 operations access and display at least two images of amusculoskeletal structure of a patient, where one pair of images are ofdifferent views of a first position or are of a same view in differentfunctional positions. At 2804 operations define reference axes of themusculoskeletal structure on at least one of the at least two images,based on user input.

At 2806 operations determine a spatial transformation between the atleast two images based on the reference axes. At 2808 operationsdetermine a positional change parameter of the musculoskeletal structureif one pair of the at least two images is the same view in differentfunctional positions, the positional change parameter representing thepositional change of the patient between the different functionalpositions. At 2810 operations render and overlay a 3D implantrespectively for each of the images in a first position relative to thereference axes and according to the spatial transformation and thepositional change parameter, if applicable. At 2812 operations renderand overlay respectively for each of the three images a safe zonegraphical element indicating a clinically accepted safe range ofpositions for the 3D implant.

A computer implemented method may provide a centering feature wherebyfor a particular safe zone, a button or other GUI interface (not shown)may be invoked to automatically position the cup such that the cup axisis in the center of the safe zone, as much as possible in each of the 2Dviews. A “best fit” may be made. The problem could be stated as acombined minimization in multiple views, and solved using linear ornon-linear optimization.

If it is not possible to center (or otherwise locate) the cup positionin each safe zone, there may be provided fixed rules to govern where thedefault cup position is defined and/or a control to provide useradjustment to a desired position. Thus the computer implemented methodmay comprise: receiving via a control interface an input toautomatically locate the 3D implant within the clinically accepted saferange of positions relative to each of the images; positioning the 3Dimplant in a safe zone position responsive to the clinically acceptedsafe range of positions in each of the images; and rendering andoverlaying the 3D implant in the safe zone position in each of theimages. It may further be that positioning the 3D implant comprisesdetermining a best fit for the safe zone position relative to each ofthe images. The computer implemented method may, for each respectiveimage of the images, update a rendering and overlaying of the safe zonegraphical element responsive to whether the safe zone position of the 3Dimplant is inside or outside the clinically accepted safe range ofpositions. The safe zone visualization may be performed along with theother methods described herein.

Thus there is provided a computer implemented method (for safe zonevisualization) comprising: accessing and displaying at least two imagesof a musculoskeletal structure of a patient, where one pair of imagesare of different views of a first position or are of a same view indifferent functional positions; defining reference axes of themusculoskeletal structure on at least one of the at least two images,based on user input; determining a spatial transformation between the atleast two images based on the reference axes; determining a positionalchange parameter of the musculoskeletal structure if one pair of the atleast two images is the same view in different functional positions, thepositional change parameter representing the positional change of thepatient between the different functional positions; rendering andoverlaying a 3D implant respectively for each of the images in a firstposition relative to the reference axes and according to the spatialtransformation and the positional change parameter, if applicable; andrendering and overlaying respectively for each of the three images asafe zone graphical element indicating a clinically accepted safe rangeof positions for the 3D implant.

The safe zone graphical element may comprise a respective graphicalelement for each of the images, each respective graphical element havingrespective graphical characteristics and wherein, for a particular imageof the images, at least one of the respective graphical characteristicsis selected based on a determination whether the 3D implant ispositioned inside or outside the clinically accepted safe range ofpositions in the particular image. The respective graphicalcharacteristics may comprise color, pattern, transparency, shape outlineand change of state comprising blinking (intensity level) and colorchange. The computer implemented method may comprise: receiving inputrepresenting a second position of the 3D implant; rendering andoverlaying accordingly the 3D implant in the second position to update,in at least near real time, each of the images respectively; determiningfor each particular image, whether the second position is inside oroutside the clinically accepted safe range of positions in theparticular image; and selecting the respective graphical characteristicsaccordingly and rendering and overlaying the respective graphicalelement for each of the images.

The computer implemented method may further comprising providingauditory and/or haptic feedback responsive to the determining when thesecond position is outside clinically accepted safe range of positions.

In the computer implemented method, rendering and overlaying the 3Dimplant may include rendering and overlaying an axis line extending fromthe 3D implant to assist with visualizing the 3D implant relative to theclinically accepted safe range of positions.

In the computer implemented method the clinically accepted safe range ofpositions may be defined in accordance with at least two angles relativeto the reference axes and an associated range of position for each ofthe at least two angles. The 3D implant may be an acetabular cup and theat least two angles are an inclination angle and an anteversion angle.

In the computer implemented method the clinically accepted safe range ofpositions may be predefined in accordance with a defined standard orselectively defined in accordance with input received. The computerimplemented method may further comprising receiving input to selectbetween the clinically accepted safe range of positions as predefined inaccordance with the defined standard or as selectively defined inaccordance with input received.

The computer implemented method may comprise: receiving via a controlinterface an input to automatically locate the 3D implant within theclinically accepted safe range of positions relative to each of the atleast two images; positioning the 3D implant in a safe zone positionresponsive to the clinically accepted safe range of positions in each ofthe at least two images; and rendering and overlaying the 3D implant inthe safe zone position in each of the at least two images. In thecomputer implemented method, positioning the 3D implant may comprisedetermining a best fit for the safe zone position relative to each ofthe at least two images. The computer implemented method may comprise,for each respective image of the at least two images, updating arendering and overlaying of the safe zone graphical element responsiveto whether the safe zone position of the 3D implant is inside or outsidethe clinically accepted safe range of positions.

In the computer implemented method the at least two images may be threeimages; wherein one pair of the three images being of different views ofa first position, and another pair of the three images being of a sameview in different functional positions; and wherein the method comprisesdetermining a positional change parameter of the musculoskeletalstructure based on the other pair showing the same view in differentfunctional positions, the positional change parameter representing thepositional change of the patient between the different functionalpositions; and wherein determining the spatial transformation betweenthe other pair utilizes the positional change parameter.

A computing system may be configured to perform the computer implementedmethod for safe zone visualization and safe zone visualization may becombined with other features and functions described herein.

Revision Surgery

The following subject matter relates to revision surgery where anexisting (first) implant is replaced with a new (second) implant.

During Revision THA, the surgeon is replacing an existing hip implantcomprising a cup with a new hip implant. In the case where the cup isstable (but is being replaced due to wear, or incorrect positioning),the orientation of the cup can be measured relative to known anatomicplanes (e.g. the APP, or the supine coronal plane for example).

FIGS. 29-31 are illustrations of a portion of a GUI 2900 showing animage 2902 of post-operative anatomy. FIG. 32 is an enlarged view of aportion of FIG. 31 . GUI 2900 may be configured as the other GUIs are,namely, with the ability to receive input, define reference axes,co-register images, define other measurements, show various overlays,etc. GUIs may present one or more images.

According to a computer implemented method, during the pre-op planningstage, a pre-op image (e.g. x-ray or other modality) may be obtained bythe computer and displayed via a display device as shown in FIG. 29 .FIG. 29 shows the current cup (CC) 2904 in a left hip as viewed from theposterior of the patient in an x-ray image.

The computer implemented method may render and overlay a 3D implant, anddisplay numerically the position of the 3D implant such as in terms ofangles (e.g. inclination and anteversion) from defined anatomical axesor planes (e.g. coronal, APP). FIG. 30 shows image 2902 comprising astanding AP x-ray with various GUI elements overlaid thereon.

One overlay is a 3D implant (a first cup implant CI1 3000). Byoverlaying the 3D implant CI1 3000 in alignment with the correspondingimplant structure of the patient in the image (e.g. current cup CC 2904in the Standing AP x-ray), the position of the corresponding implantstructure (current cup CC 2904) relative to an axis may be determinedfrom the position of the 3D implant. The measurement may be displayedsuch as in a GUI element for a numerical display (ND1 3002). A graphicaluser interface may be configured to provide object handles for the 3Dimplant overlay with which to move the 3D implant (overlay) CI1 3000 andalign it with the corresponding implant structure or otherwise move it.In FIG. 30 , when viewed in color, the 3D implant CI1 is blue and showsthe visual position (over the current cup CC 2904 in the radiograph).The measurements in ND1 3002 show the angles (i.e. a current position)of the 3D implant CI1 3000. When overlaid on the cup CC 2904 thesenumbers represent the measurement of the corresponding implant structurein the patient. An inclination reference 3004 (IR) (e.g. axis for theimage) may be established (e.g. inputs received to determine end points)and displayed.

Measuring the inclination and anteversion may be performed according toone or more methods. For example, a computer implemented method mayprovide a 3D implant object with handles (as shown elsewhere herein) andas shown in FIG. 31 for a user to manually fit (e.g. position and size)the 3D implant appropriately over the current cup CC in one image or twoor more co-registered images. Two of the co-registered images made ofthe same functional position from different views (e.g. standing AP andstanding lateral views). Controls to zoom in or zoom out the display, toenlarge or shrink image size and 3D implant size, may assist withappropriate resolution to permit a user to fit the 3D implant. In a moreautomatic fashion, for example, a computer implemented method mayautomatically fit a 3D implant object to a current cup CC such as byusing image processing to determine the location of the current cup CCin one image or two or more co-registered images. It may use edgedetection and determine a cup face and the applicable pose as definedrelative to inclination and anteversion. Automatic and manual methodsmay be combined or provided as options. A 3D implant may beautomatically fit and a user permitted to manually adjust the fit.

The computer implemented method may provide a control (e.g. a button, amenu item, a voice activated control) to receive input to invoke arevision tool. The GUI may be configured to display an additionalgraphical element for numerical display (e.g. ND2 3100) such as shown inFIG. 31 . The additional graphical element may be a second numericaldisplay to show a second set of measurements as described further. TheGUI may also show a “Tare” button (TB) 3102 (or other control interface)to receive input to establish a reference cup position (similar totaring a scale). When the tare button is invoked, the current position(absolute angles 40° inclination and 20° anteversion) of the 3D implantCI1 3000 is stored. Visually, a further graphical element representingthe 3D implant (e.g. a second 3D implant overlay CI2 3104) may bedisplayed and maintained in the reference cup position. When viewed incolor the overly CI2 3104 may be maroon in color. Though not shown, acontrol may be provided to turn the display of CI2 3104 on or off. CI2may be distinguished from CI1 in the GUI such as by a difference ofcolor, line style, shape or other appearance characteristics.

The position of the first 3D implant overlay CI1 3000 may be moved to asecond position such as via one or more handles (H1 3108A, H2 3108B andH3 3108C) or via input to the numerical display ND1 3002 to adjust oneor both of the angles. Input to adjust angles (measurement data) mayinvoke the GUI to update the position of CI1 3000 (to render and overlayit in a second position) in the display over the patient image 2902 toaccord with the input. Input to the numerical display may comprisereceiving a specific angle number (e.g. 22) or a control input to raiseor lower the present number (e.g. to add or subtract 1° via a respectiveplus/minus or other control (not shown)). The computer implementedmethod may update the displaying of the current position (e.g. in ND13002) in response to a movement of the 3D implant (e.g. via the handles)such as in real time.

FIG. 31 shows CI1 3000 in a second position where the cup inclinationhas decreased by 18 degrees, and the cup anteversion has increased by 7degrees. The numerical display elements ND1 3002 and ND2 3100 maypresent the absolute angle references (ND1 3000) of CI1 3000 and thechange or delta measurement) in such measurements for CI1 3000 relativeto the reference cup position previously saved using the tare button TB.Thus the computer implemented method may determine and store a deltameasurement comprising a difference between the reference position andthe second position and display this delta measurement.

As shown in FIG. 31 , GUI 2900 provides both a visual indication of thechange in cup position by showing two overlays (with CI2 3104 in thereference position (maroon) vs CI1 3000 in the second position (cyan)),and the cup orientation numbers in a relevant coordinate system.

The output of the planning session for a revision surgery (as providedby the computer implemented method) is now a “delta” target, instead ofan absolute cup position target. Such data may be saved to a memory (orother storage device) and/or communicated to another computer apparatus,etc. for use during a surgical procedure such as one involving anintraoperative surgical navigation or localization system. In oneexample, the data may be encoded such as in a matrix barcode (e.g. aQuick Response (QR) code—QR Code is a registered trademark in the UnitedStates of Denso Wave Incorporated) and communicated such as via adisplay device or printout.

This target may be communicated to an intraoperative surgical navigationsystem which has the ability to measure relevant anatomic landmarks(such as coronal plane, APP, etc.) to be used for anatomic referenceaxes, and has the ability to measure the position of an existing implantrelative to this registration. When the reference axes of the surgicalnavigation system are the same as the reference axes of the planningsoftware, and the subject of the measurement (i.e. the patient) is thesame, the communicated target can be received by the surgical navigationsystem and, in conjunction with the intra-operative measurement, providean intra-operative target which minimizes error due to discrepancies inthe reference axes, such discrepancies possibly arising from factorssuch as soft tissue, mis-probing, incorrect patient positioning, etc.

Since absolute measurements in a respective intraoperative surgicalnavigation or localization system and method to assist with surgery anda respective surgical planning computer system and method are maderelative to their respective coordinate systems, such coordinate systemsmust be aligned for the absolute measurements to be related. However, bydetermining and utilizing a change in absolute measurements (i.e.“delta” measurements in both the planning and intraoperative surgicalsystems), the differences in the two coordinate systems may be negated.The relationship between the “delta” measurements in a planning systemand method and the “delta” measurements in the intraoperative system andmethod are less affected by discrepancies between the coordinate systemsof the respective applications.

The “delta” target has been described above as being communicated in ananatomically relevant coordinate system, e.g. radiographicinclination/anteversion relative to the standing coronal plane. Byrelying on a “delta” instead of absolute measurements, smalldiscrepancies in reference axes between planning software and surgicalnavigation software can have minimal impact on final implantpositioning. The target could instead by described and communicated in areference coordinate system relative to the existing implant itself, orrelative to a combination of the implant and anatomic measurements. Thiscould reduce any discrepancies between reference axes in the respectiveplanning and navigation software due to the fact that implants are moreaccurately measured in both software/systems than are anatomicallandmarks.

In one example, a “delta” target for an acetabular implant (representedas a hemisphere), could be described with two angles of change: a thetaangle which represents the angle between the existing and target cupnormal vectors, and a phi angle representing the rotation of this changerelative to the transverse axis of the patient. Normal vectors arerelative to the face of each respective implant. FIG. 32A represents cuphemispheres 3200 and 3202 for the existing implant and a planned implantrespectively. In FIG. 32A, the existing implant is represented as thehemisphere EI 3200 (having a broken line on the outer margin of the cupface) with a normal line EINL 3206 (also shown broken in style), and theplanned implant 3202 is represented as the hemisphere PI 3202 with anormal line PINL 3208. FIG. 32A shows how theta (θ) is the absoluteangle between the normal of the planned implant PINL 3208 and the normalof the existing cup EINL 3206.

FIG. 32B shows the same cups as hemispheres 3200 and 3204 from aposition perpendicular to the existing cup face (the existing cup normalvector EINL 3206 points towards reader) and phi (φ) is the angle betweenthe red transverse axis TA 3210 of the body, and the planned cup vectorPINL 3208, when both are projected onto the face of the cup. Thecomputer implemented method may be configured to generate the deltameasurements relative to the existing implant. With reference to theFIG. 30 and FIG. 31 , a 3D implant CI1 3000 is overlaid on the currentcup 2904 (existing implant). A reference position is established such asby invoking the Tare control 3102. A second 3D implant CI2 3104 isoverlaid to mark the reference position. The 3D implant overlay CI1 3000is moved to the planned input position (see FIG. 31 ). Values theta andphi may be determined between the reference position and the plannedposition, for example, using the model of the 3D implant (to determinethe plane of the cup face and any normal thereto) and its two locationsin the coordinate system of the radiographic system. Determining thesetwo angles determines the changes for the planned implant relative tothe existing implant without requiring an association with thecoordinate system of the radiographic image.

Though only one radiographic image is shown in FIGS. 30 and 31 , twoco-registered views (or more) may be shown such as illustrated elsewhereherein. Movement of CI1 3000 in a particular view (e.g. such as bymanual adjustment via handles, etc.) may be reflected in the otherview(s) accordingly such as described elsewhere herein. A suitablespatial transformation (more than one) may be determined forappropriately rendering and overlaying the 3D implant and any saveoriginal position as represented by CI2 3104.

FIG. 34 is a flowchart of operations 3400 for a computing device such asthe computing device 900. Operations 3400 may provide a method forpreoperative planning for revision surgery.

At 3402 operations access and display an image of a musculoskeletalstructure of a patient and displaying the image via a display device. At3404 operations define a reference axis of the musculoskeletal structure(e.g. from input received or as received from previous computation). At3406 operations render and overlay a 3D implant in a first position overthe image, the first position defined relative to the reference axis. At3408 operations receive input to define a second position for the 3Dimplant, the second position defined relative to the reference axis. At3410 operations render and overlay the 3D implant in association withthe second position over the image. At 3412 operations determine andstore a delta measurement comprising a difference between the firstposition and the second position; and at 3414 operations provide thedelta measurement for display via the display device.

Thus, there is provided a computer implemented method (e.g. for planninga revision surgery) comprising: accessing an image of a musculoskeletalstructure of a patient and displaying the image via a display device;defining a reference axis of the musculoskeletal structure; renderingand overlaying a 3D implant in a first position over the image, thefirst position defined relative to the reference axis; receiving inputto define a second position for the 3D implant, the second positiondefined relative to the reference axis; rendering and overlaying the 3Dimplant in association with the second position over the image;determining and storing a delta measurement comprising a differencebetween the first position and the second position; and providing thedelta measurement for display via the display device.

The computer implemented may comprise receiving input to store the firstposition as a reference position for the 3D implant in a storage device.The computer implemented method may comprise rendering and overlaying afurther graphical element representing the 3D implant in the referenceposition to distinguish the 3D implant as overlaid in the secondposition.

In the computer implemented method the 3D implant may be overlaid inalignment with a corresponding implant structure of the patient in theimage and wherein the first position provides a measurement of thecorresponding implant structure relative to the reference axis.

The computer implemented method may comprise displaying, via the displaydevice, a current position of the 3D implant relative to the referenceaxis. The computer implemented may comprise updating the displaying ofthe current position in response to a movement of the 3D implant.

The computer implemented method may comprise providing a graphical userinterface (GUI) to display handles for the 3D implant and to receiveinput to move the 3D implant over the image using the handles. Thecomputer implemented method may comprise receiving input to define thefirst position via the handles for the 3D implant. The input may definea fit for the 3D implant over an implant in the patient visible in theimage.

The computer implemented method may comprise fitting the 3D implant overan implant in the patient visible in the image using image processing.

The computer implemented method may comprise providing a GUI to receivemeasurement data relative to the reference axis and rendering andoverlaying the 3D implant in accordance with the measurement data.

In the computer implemented method, the musculoskeletal structure maycomprise a hip structure and the 3D implant comprises a cup implant.

In the computer implemented method the first position and the secondposition may be defined in accordance with an inclination angle and ananteversion angle.

The computer implemented method may be performed preoperatively using acomputer apparatus configured to provide a preoperative planning systemand wherein the method may comprise communicating the delta measurementto a computing apparatus configured to provide an intraoperativesurgical system to facilitate a surgical procedure.

In the computer implemented, the image may comprise either: apreoperative image displayed preoperatively; or an intraoperative imagedisplayed intraoperatively.

The computer implemented method may comprise determining the deltameasurement as angles of change relative to an existing implant in thepatient as represented by the 3D implant in the first position and the3D implant in the second position.

The computer implemented method may comprise communicating the deltameasurement for further processing. The further processing may besurgical navigation on the patient in the image using an intra-operativesurgical navigation system, and the first position may be the same as ameasured implant position for an implant in the patient as measured bythe intra-operative surgical navigation system and the intra-operativesurgical navigation system may be configured to measure a change to animplant position relative to the measured implant position.

In the computer implemented method the image may define a first imageand the method may comprise: accessing a second image of themusculoskeletal structure of the patient and displaying the second imagevia the display device together with the first image; determining aspatial transformation between the first image and the second image;rendering and overlaying the 3D implant in the first position over thesecond image based on the reference axis and the spatial transformationto display the 3D implant respectively on both of the first image andthe second image; and, in response to receiving input to define thesecond position, rendering and overlaying the 3D implant in associationwith the second position over the second image. The computer implementedmethod may comprise providing handles to interact with the 3D implant asoverlaid on each of the first image and the second image and wherein theinput to define the second position for the 3D implant comprisesreceiving input via the handles as provided on one of the first imageand the second image. In the computer implemented method the first imageand the second image may comprise different views of the same functionalposition.

Conclusion

The methods herein may be performed preoperatively or intraoperatively.That is, the image displayed may comprise either: a preoperative image(e.g. static x-ray or other image generated prior to an operation whichis displayed preoperatively; or may be an image generated during aprocedure (“an intraoperative image”) that is displayed intraoperativelyand the method to determine the delta measurements undertakenintraoperatively. Planning as described herein may include planningduring a procedure, for example, in an operating room as well as priorto initiating a procedure in an operating room.

Additionally or alternatively to any GUI interface options or controlsdiscussed, voice activated controls may be provided.

In addition to computing device aspects, a person of ordinary skill willunderstand that computer program product aspects are disclosed, whereinstructions are stored in a non-transient storage device (e.g. amemory, CD-ROM, DVD-ROM, disc, etc.) to configure a computing device toperform any of the method aspects stored herein.

Practical implementation may include any or all of the featuresdescribed herein. These and other aspects, features and variouscombinations may be expressed as methods, apparatus, systems, means forperforming functions, program products, and in other ways, combining thefeatures described herein. A number of embodiments have been described.Nevertheless, it will be understood that various modifications can bemade without departing from the spirit and scope of the processes andtechniques described herein. In addition, other steps can be provided,or steps can be eliminated, from the described process, and othercomponents can be added to, or removed from, the described systems.Accordingly, other embodiments are within the scope of the followingclaims.

Throughout the description and claims of this specification, the word“comprise” and “contain” and variations of them mean “including but notlimited to” and they are not intended to (and do not) exclude othercomponents, integers or steps. Throughout the description and claims ofthis specification, singular encompasses the plural unless the contextrequires otherwise. In particular, where the indefinite article is used,the specification is to be understood as contemplating plurality as wellas singularity, unless the context requires otherwise.

Features, integers characteristics, compounds, chemical moieties orgroups described in conjunction with a particular aspect, embodiment orexample of the invention are to be understood to be applicable to anyother aspect, embodiment or example unless incompatible therewith. Allof the features disclosed herein (including any accompanying claims,abstract and drawings), and/or all of the steps of any method or processso disclosed, may be combined in any combination, except combinationswhere at least some of such features and/or steps are mutuallyexclusive. The invention is not restricted to the details of anyforegoing examples or embodiments. The invention extends to any novelone, or any novel combination, of the features disclosed in thisspecification (including any accompanying claims, abstract and drawings)or to any novel one, or any novel combination, of the steps of anymethod or process disclosed.

The invention claimed is:
 1. A computer system comprising at least oneprocessing unit and a memory coupled to the at least one processingunit, a storage device storing instructions that, when executed by theat least one processing unit, cause the computer system to: access atleast three images of a musculoskeletal structure of a patient anddisplay the at least three images via a display device; wherein a firstpair of images are of different views of a first functional position ofa musculoskeletal structure of a patient, and a second pair of imagesare of the same view of different functional positions of amusculoskeletal structure of a patient; define at least one referenceaxis of the musculoskeletal structure on at least one of the images;determine a positional change parameter of the musculoskeletal structurebased on the second pair of images, the positional change parameterrepresenting the positional change of the patient between the differentfunctional positions of the second pair of images; determine a firstspatial transformation between the images of the first pair of imagesand determine a second spatial transformation between the images of thesecond pair of images; render and overlay a three dimensional (3D)implant for each image in a first implant position defined relative tothe reference axes, and according to the first spatial transformationfor the first pair of images and according to the second spatialtransformation and positional change parameter for the second pair ofimages, and overlay the 3D implant on the respective images; receiveinput to define a second implant position for the 3D implant, the secondimplant position defined relative to the reference axis; and update, inat least near real time, the rendering and overlay accordingly for eachimage in the second implant position using each spatial transformationand positional change parameter.
 2. The computer system of claim 1,wherein a modality of the at least three images comprises any of: x-ray,computed tomography (CT), magnetic resonance imaging (MRI), and EOSimaging modalities.
 3. The computer system of claim 1, wherein thecomputer system is further configured to receive user input; and whereindefining at least one reference axis is based on user input via one ormore of: a selection of one or more coordinates defining the referenceaxes on one or more of the at least three images; and an inputting ofnumerical values.
 4. The computer system of claim 1, wherein thecomputer system is further configured to display the positional changeparameter.
 5. The computer system of claim 1, wherein the computersystem is configured to determine the first and second spatialtransformations by one or more of: performing computations based atleast in part on axes coordinates received on one or more images asinput to define the at least one reference axis of the musculoskeletalstructure on the one or more images; performing computations based oncorresponding common features between two or more of the at least threeimages; and performing computations based on known image acquisitionspatial information.
 6. The computer system of claim 1, wherein thecomputer system is further configured to: receive templating informationcomprising one or more of implant size and desired location; and renderthe 3D implant according to at least a subset of the received templatinginformation.
 7. The computer system of claim 1, wherein themusculoskeletal structure is a pelvis, the 3D implant is an acetabularcup and the first implant position is 40 degrees of inclination and 15degrees of anteversion.
 8. The computer system of claim 1, wherein thecomputer system is further configured to display position informationrepresenting a current position of the 3D implant; wherein the 3Dimplant is an acetabular cup and the position information is aninclination angle and an anteversion angle.
 9. The computer system ofclaim 1, wherein the input representing the second position of the 3Dimplant is based on one or more of: a manipulation of an imageassociated with the 3D implant as overlaid on one of the at least threeimages of a musculoskeletal structure of a patient; and an inputting ofnumerical values.
 10. The computer system of claim 9 wherein themanipulation of the image comprises clicking and dragging a handleassociated with the 3D implant overlay.
 11. The computer system of claim1, wherein the first pair of images comprises a standing anteroposterior(AP) image and a standing lateral image, and the second pair of imagescomprises a standing lateral image and a sitting lateral image.
 12. Thecomputer system of claim 11, wherein the positional change parameter isa tilt parameter; and wherein a change in tilt parameter between thestanding AP image and the standing lateral image is determined based onrespective tilt coordinates received on the standing lateral image andthe sitting lateral image respectively via user input.
 13. Acomputer-implemented method comprising: accessing at least three imagesof a musculoskeletal structure of a patient and displaying the at leastthree images via a display device; wherein a first pair of images are ofdifferent views of a first functional position of a musculoskeletalstructure of a patient, and a second pair of images are of the same viewof different functional positions of a musculoskeletal structure of apatient; defining at least one reference axis of the musculoskeletalstructure on at least one of the images; determining a positional changeparameter of the musculoskeletal structure based on the second pair ofimages, the positional change parameter representing the positionalchange of the patient between the different functional positions of thesecond pair of images; determining a first spatial transformationbetween the images of the first pair of images and determining a secondspatial transformation between the images of the second pair of images;rendering and overlaying a three dimensional (3D) implant for each imagein a first implant position defined relative to the reference axes, andaccording to the first spatial transformation for the first pair ofimages and to the second spatial transformation and positional changeparameter for the second pair or images, and overlay the 3D implant onthe respective images; receiving input to define a second implantposition for the 3D implant, the second implant position definedrelative to the reference axis; and update, in at least near real time,the rendering and overlay accordingly for each image in the secondimplant position using each spatial transformation and positional changeparameter.
 14. The computer-implemented method of claim 13 wherein amodality of the at least three images comprises any of: x-ray, computedtomography (CT), magnetic resonance imaging (MRI), and EOS imagingmodalities.
 15. The computer-implemented method of claim 13 furthercomprising receiving user input; and wherein defining at least onereference axis is based on user input via one or more of: a selection ofone or more coordinates defining the reference axes on one or more ofthe at least three images; and an inputting of numerical values.
 16. Thecomputer-implemented method of claim 13, wherein determining the firstand second spatial transformations comprise one or more of: performingcomputations based at least in part on axes coordinates received on oneor more images as input to define the at least one reference axis of themusculoskeletal structure on the one or more images; performingcomputations based on corresponding common features between two or moreof the at least three images; and performing computations based on knownimage acquisition spatial information.
 17. The computer-implementedmethod of claim 13, wherein the input representing the second positionof the 3D implant is based on one or more of: a manipulation of an imageassociated with the 3D implant as overlaid on one of the at least threeimages of a musculoskeletal structure of a patient; wherein themanipulation of the image comprises clicking and dragging a handleassociated with the 3D implant overlay; and an inputting of numericalvalues.
 18. The computer-implemented method of claim 13, wherein thefirst pair of images comprises a standing anteroposterior (AP) image anda standing lateral image, and the second pair of images comprises astanding lateral image and a sitting lateral image.
 19. Thecomputer-implemented method of claim 18, wherein the positional changeparameter is a tilt parameter; and wherein a change in tilt parameterbetween the standing AP image and the standing lateral image isdetermined based on respective tilt coordinates received on the standinglateral image and the sitting lateral image respectively via user input.20. A computer system comprising at least one processing unit and amemory coupled to the at least one processing unit, a storage devicestoring instructions that, when executed by the at least one processingunit, cause the computer system to: access a standing anteroposterior(AP) image, a standing lateral image, and a sitting lateral image of amusculoskeletal structure of a patient and display the images via adisplay device; receive respective axes coordinates on the standing APimage and standing lateral image respectively as input, and definereference axes of the musculoskeletal structure on the standing AP imageand the standing lateral image based on the respective axes coordinates;receive respective tilt coordinates on the standing lateral image andthe sitting lateral image respectively as input, and determine a changein tilt parameter between the standing AP image and the standing lateralimage of the musculoskeletal structure based on the respective tiltcoordinates; determine a spatial transformation between the standing APimage and the standing lateral image and a spatial transformationbetween the standing lateral image and the sitting lateral image; renderand overlay a three dimensional (3D) implant for each of the standing APimage and the standing lateral image in a first position relative to thereference axes based on the reference axes and the spatialtransformation between the standing AP image and the standing lateralimage; render and overlay the 3D implant in the first position for thesitting lateral image based on the change in tilt parameter and thespatial transformation between the standing lateral image and thesitting lateral image; receive input representing a second position ofthe 3D implant; and render and overlay the 3D implant in the secondposition using each space transformation and the change in tiltparameter to update, in real time, each of the standing AP image, thestanding lateral image and the sitting lateral image.